Acknowledgment of patient in sense behaviors using bidirectional ConvLSTM

Recognizing patient activity in real-time from video or images collected by a CCTV camera available in the hospital during a Covid-19 situation has proven challenging. The dilemma of patient activity recognition is identifying and recognizing a patient's various actions in a series of videos. The process presented in our paper needs to achieve unrestricted, generic behavior in videos. Detecting events in any video is often difficult because we use Bidirectional ConvLSTM to create a robust patient in the sense behaviors (PSB) framework capable of eliminating certain barriers. To begin this paper by proposing a new Bidirectional ConvLSTM for establishing a stable PSB scheme. Our proposed model is capable of accurately predicting patient's behaviors like seated, standing, and so on. Using Bidirectional ConvLSTM, learning information from a pre-trained model is an excellent place to start for rapidly developing a new PSB system using a current PSB database, as both the source and target datasets are critical. All parameters are frozen in a pre-trained PSB device. Then, using the UCI and HMDB51 dataset to train the model, variables and local relations are progressively fixed. A novel PSB framework is developed using the target dataset. Relevant tests are conducted using commonly used research indices to assess prediction precision accuracy. They acknowledge six patient's behavior with a weighted accuracy rate of 92%. For recognizing novel activity, laying, the precision of a corresponding prediction is the best, 91%, of all six test results. The proposed work uses bidirectional ConvLSTM with modified activation layers to sense the patients' behavior. This article may be a patient activity recognition system to identify an individual. It takes a clip of COVID-19 patients as input and looks for matches inside the hold-on images.

Real-time Fitness Activity Recognition and Correction using Deep Neural Networks

Fitness activities are beneficial to one's health and well-being. During the Covid-19 pandemic, demand for virtual trainers increased. There are current systems that can classify different exercises, and there are other systems that provide feedback on a specific exercise. We propose a system that can simultaneously recognize a pose as well as provide real-time corrective feedback on the performed exercise with the least latency between recognition and correction. In all computer vision techniques implemented so far, occlusion and a lack of labeled data are the most significant problems in correctly detecting and providing helpful feedback. Vector geometry is employed to calculate the angles between key points detected on the body to provide the user with corrective feedback and count the repetitions of each exercise. Three different architectures-GAN, Conv-LSTM, and LSTM-RNN are experimented with, for exercise recognition. A custom dataset of Jumping Jacks, Squats, and Lunges is used to train the models. GAN achieved a 92% testing accuracy but struggled in real-time performance. The LSTM-RNN architecture yielded a 95% testing accuracy and ConvLSTM obtained an accuracy of 97% on real-time sequences. © 2023 IEEE.

Individual Surveillance Around Parked Aircraft at Nighttime: Thermal Infrared Vision-Based Human Action Recognition

The COVID-19 crisis has led to an unusually large number of commercial aircraft being currently parked or stored. For airlines, airports, and civil aviation authorities around the world, monitoring, and protecting these parked aircraft to prevent them from causing human-made damage are becoming urgent problems that are receiving increasing attention. In this study, we use thermal infrared monitoring videos to establish a framework for individual surveillance around parked aircraft by proposing a human action recognition (HAR) algorithm. As the focus of this article, the proposed HAR algorithm seamlessly integrates a preprocessing module in which a novel data structure is constructed to introduce spatiotemporal information of the action;a convolutional neural network-based module for spatial feature extraction;a triple-layer convolutional long short-term memory network for temporal feature extraction;and two fully connected layers for classification. Moreover, because no infrared dataset is available for the HAR task on airport grounds at nighttime, we present a dataset called IIAR-30, which consists of eight action categories that frequently occur on airport grounds and 2000 video clips. The experimental results on the IIAR-30 dataset demonstrated that the recognition accuracy of the proposed method was higher than 96%. We also further evaluated the effectiveness of the proposed method by comparing it with five baselines and four other methods. © 2022 IEEE.

Individual Surveillance Around Parked Aircraft at Nighttime: Thermal Infrared Vision-Based Human Action Recognition

The COVID-19 crisis has led to an unusually large number of commercial aircraft being currently parked or stored. For airlines, airports, and civil aviation authorities around the world, monitoring, and protecting these parked aircraft to prevent them from causing human-made damage are becoming urgent problems that are receiving increasing attention. In this study, we use thermal infrared monitoring videos to establish a framework for individual surveillance around parked aircraft by proposing a human action recognition (HAR) algorithm. As the focus of this article, the proposed HAR algorithm seamlessly integrates a preprocessing module in which a novel data structure is constructed to introduce spatiotemporal information of the action;a convolutional neural network-based module for spatial feature extraction;a triple-layer convolutional long short-term memory network for temporal feature extraction;and two fully connected layers for classification. Moreover, because no infrared dataset is available for the HAR task on airport grounds at nighttime, we present a dataset called IIAR-30, which consists of eight action categories that frequently occur on airport grounds and 2000 video clips. The experimental results on the IIAR-30 dataset demonstrated that the recognition accuracy of the proposed method was higher than 96%. We also further evaluated the effectiveness of the proposed method by comparing it with five baselines and four other methods.

Hyper-parameter Tuning for Long Short-Term Memory (LSTM) Algorithm to Forecast a Disease Spreading

Artificial intelligence (AI), machine learning (ML), and deep learning (DL) are just a few of the technologies that can help healthcare organizations support real-time decision-making to control the spread of a pandemic. This study aims to investigate hyper-parameter tuning for Long Short-Term Memory (LSTM) to forecast SARSCoV2 daily infection cases in the Russian Federation by selecting the best loss function, activation function, number of epochs, number of neurons in a cell, and optimizer to minimize the error in addition to a good fit for the model on both the training and validation sets. In the meanwhile, we use LSTM, LSTMs (stacked LSTM), bidirectional LSTM, BILSTMs (stacked BILSTM), convolution neuron networks (Conv), ConvLSTMs and other forecasting models to forecast the daily SARSCoV2 infection cases one by one. We analyzed and compared the results obtained by these models. We discovered that BiLSTM can efficiently extract features from the data, which are the items from the previous day. With the extracted feature data, we used this model to forecast daily infection cases. We used BiLSTM to forecast SARSCoV2 spreading in the Russian Federation for one month. Hence, the Bidirectional LSTM can accurately predict daily SARSCoV2 infection cases in Russia, according to experiments. © 2022 IEEE.

Comparison of Nitrogen Dioxide Predictions During a Pandemic and Non-pandemic Scenario in the City of Madrid using a Convolutional LSTM Network

Traditionally, machine learning technologies with the methods and capabilities available, combined with a geospatial dimension, can perform predictive analyzes of air quality with greater accuracy. However, air pollution is influenced by many external factors, one of which has recently been caused by the restrictions applied to curb the relentless advance of COVID-19. These sudden changes in air quality levels can negatively influence current forecasting models. This work compares air pollution forecasts during a pandemic and non-pandemic period under the same conditions. The ConvLSTM algorithm was applied to predict the concentration of nitrogen dioxide using data from the air quality and meteorological stations in Madrid. The proposed model was applied for two scenarios: pandemic (January–June 2020) and non-pandemic (January–June 2019), each with sub-scenarios based on time granularity (1-h, 12-h, 24-h and 48-h) and combination of features. The Root Mean Square Error was taken as the estimation metric, and the results showed that the proposed method outperformed a reference model, and the feature selection technique significantly improved the overall accuracy.

Individual Surveillance Around Parked Aircraft at Nighttime: Thermal Infrared Vision-Based Human Action Recognition

The COVID-19 crisis has led to an unusually large number of commercial aircraft being currently parked or stored. For airlines, airports, and civil aviation authorities around the world, monitoring, and protecting these parked aircraft to prevent them from causing human-made damage are becoming urgent problems that are receiving increasing attention. In this study, we use thermal infrared monitoring videos to establish a framework for individual surveillance around parked aircraft by proposing a human action recognition (HAR) algorithm. As the focus of this article, the proposed HAR algorithm seamlessly integrates a preprocessing module in which a novel data structure is constructed to introduce spatiotemporal information of the action;a convolutional neural network-based module for spatial feature extraction;a triple-layer convolutional long short-term memory network for temporal feature extraction;and two fully connected layers for classification. Moreover, because no infrared dataset is available for the HAR task on airport grounds at nighttime, we present a dataset called IIAR-30, which consists of eight action categories that frequently occur on airport grounds and 2000 video clips. The experimental results on the IIAR-30 dataset demonstrated that the recognition accuracy of the proposed method was higher than 96%. We also further evaluated the effectiveness of the proposed method by comparing it with five baselines and four other methods.

A deep learning approach for Spatio-Temporal forecasting of new cases and new hospital admissions of COVID-19 spread in Reggio Emilia, Northern Italy.

BACKGROUND: Since February 2020, the COVID-19 epidemic has rapidly spread throughout Italy. Some studies showed an association of environmental factors, such as PM10, PM2.5, NO2, temperature, relative humidity, wind speed, solar radiation and mobility with the spread of the epidemic. In this work, we aimed to predict via Deep Learning the real-time transmission of SARS-CoV-2 in the province of Reggio Emilia, Northern Italy, in a grid with a small resolution (12 km × 12 km), including satellite information. METHODS: We focused on the Province of Reggio Emilia, which was severely hit by the first wave of the epidemic. The outcomes included new SARS-CoV-2 infections and COVID-19 hospital admissions. Pollution, meteorological and mobility data were analyzed. The spatial simulation domain included the Province of Reggio Emilia in a grid of 40 cells of (12 km)2. We implemented a ConvLSTM, which is a spatio-temporal deep learning approach, to perform a 7-day moving average to forecast the 7th day after. We used as training and validation set the new daily infections and hospital admissions from August 2020 to March 2021. Finally, we assessed the models in terms of Mean Absolute Error (MAE) compared with Mean Observed Value (MOV) and Root Mean Squared Error (RMSE) on data from April to September 2021. We tested the performance of different combinations of input variables to find the best forecast model. FINDINGS: Daily new cases of infection, mobility and wind speed resulted in being strongly predictive of new COVID-19 hospital admissions (MAE = 2.72 in the Province of Reggio Emilia; MAE = 0.62 in Reggio Emilia city), whereas daily new cases, mobility, solar radiation and PM2.5 turned out to be the best predictors to forecast new infections, with appropriate time lags. INTERPRETATION: ConvLSTM achieved good performances in forecasting new SARS-CoV-2 infections and new COVID-19 hospital admissions. The spatio-temporal representation allows borrowing strength from data neighboring to forecast at the level of the square cell (12 km)2, getting accurate predictions also at the county level, which is paramount to help optimise the real-time allocation of health care resources during an epidemic emergency.

CovidConvLSTM: A fuzzy ensemble model for COVID-19 detection from chest X-rays.

The rapid outbreak of COVID-19 has affected the lives and livelihoods of a large part of the society. Hence, to confine the rapid spread of this virus, early detection of COVID-19 is extremely important. One of the most common ways of detecting COVID-19 is by using chest X-ray images. In the literature, it is found that most of the research activities applied convolutional neural network (CNN) models where the features generated by the last convolutional layer were directly passed to the classification models. In this paper, convolutional long short-term memory (ConvLSTM) layer is used in order to encode the spatial dependency among the feature maps obtained from the last convolutional layer of the CNN and to improve the image representational capability of the model. Additionally, the squeeze-and-excitation (SE) block, a spatial attention mechanism, is used to allocate weights to important local features. These two mechanisms are employed on three popular CNN models - VGG19, InceptionV3, and MobileNet to improve their classification strength. Finally, the Sugeno fuzzy integral based ensemble method is used on these classifiers' outputs to enhance the detection accuracy further. For experiments, three chest X-ray datasets, which are very prevalent for COVID-19 detection, are considered. For all the three datasets, it is found that the results obtained by the proposed method are comparable to state-of-the-art methods. The code, along with the pre-trained models, can be found at https://github.com/colabpro123/CovidConvLSTM.

Short-Term Traffic Forecasting using LSTM-based Deep Learning Models

Accurate short-term traffic volume forecasting has become a component with growing importance in traffic management in intelligent transportation systems (ITS). A significant amount of related works on short-term traffic forecasting has been proposed based on traditional learning approaches, and deep learning-based approaches have also made significant strides in recent years. In this paper, we explore several deep learning models that are based on long-short term memory (LSTM) networks to automatically extract inherent features of traffic volume data for forecasting. A simple LSTM model, LSTM encoder-decoder model, CNN-LSTM model and a Conv-LSTM model were designed and evaluated using a real-world traffic volume dataset for multiple prediction horizons. Finally, the experimental results are analyzed, and the Conv-LSTM model produced the best performance with a MAPE of 9.03% for the prediction horizon of 15 minutes. Also, the paper discusses the behavior of the models with the traffic volume anomalies due to the Covid-19 pandemic.

Convolutional LSTM Network for Forecasting Correlations Between Stocks Based on Spatiotemporal Sequence

The correlation between stocks is important for investment portfolio pricing and evaluation, risk management, and formulating trading and hedging strategies. The COVID-19 has led to a general increase in the degree of correlation between stocks, the market-wide allocation has lost its meaning, and the hedging strategy has failed. It is more necessary and urgent to predict the correlation between stocks under the influence of the epidemic. However, previous studies mostly focused on traditional financial models. There are problems such as too many assumptions and restrictions, the dimensional disaster of the estimated parameters, and the poor effect of fitting nonlinearity and tail risk, which cannot provide reliable and accurate estimates. In this paper, the covariance matrix for stock return is considered as a sequence with both time and space characteristics, to transform the problem into the study of spatiotemporal sequence prediction. We Innovatively apply the end-to-end Convolutional LSTM (ConvLSTM) to the correlation prediction between stocks and use random matrix theory (RMT) to improve mean squared error (MSE) to eliminate the influence of noise. Experiments show that the performance of ConvLSTM on this problem is better than that of traditional financial model, especially after de-nosing by Random Matrix Theory (RMT). Compared with Fully Connected LSTM (FC-LSTM), ConvLSTM acquired a better out-of-sample MSE and RMT_MSE, which proves the effectiveness of the method. Finally, we repeat experiments with other stock dataset to verify the robustness of the model. © 2021 IEEE.

A Deep Convolutional LSTM for ADLs Classification of the Elderly

Activity analysis systems or activity recognition systems for the elderly is recently a part of the smart home systems design. This assisted system normally helps the elderly to live alone in a house, safely and improve a quality of life. Therefore, learning to recognize which activities are safe is necessary for classifying the activity of the elderly. Furthermore, this information will give us some insights to understand the basic daily lives of the elderly and it also helps us to monitor activity and health information of the elderly. In this paper, we collected activities data using the multi-sensor motion sensors embedded inside the smartwatch (Fitbit). We also present the novel method for detecting and recognizing the activity using Deep Convolutional LSTM. In brief, this paper shows that the proposed method yields 88.425% of accuracy for activity classification. The paper also compares the results with our previous work which used Backpropagation Neural Networks as a classifier (78% of accuracy). © 2021 IEEE.

Automatic Sequence-Based Network for Lung Diseases Detection in Chest CT.

OBJECTIVE: To develop an accurate and rapid computed tomography (CT)-based interpretable AI system for the diagnosis of lung diseases. BACKGROUND: Most existing AI systems only focus on viral pneumonia (e.g., COVID-19), specifically, ignoring other similar lung diseases: e.g., bacterial pneumonia (BP), which should also be detected during CT screening. In this paper, we propose a unified sequence-based pneumonia classification network, called SLP-Net, which utilizes consecutiveness information for the differential diagnosis of viral pneumonia (VP), BP, and normal control cases from chest CT volumes. METHODS: Considering consecutive images of a CT volume as a time sequence input, compared with previous 2D slice-based or 3D volume-based methods, our SLP-Net can effectively use the spatial information and does not need a large amount of training data to avoid overfitting. Specifically, sequential convolutional neural networks (CNNs) with multi-scale receptive fields are first utilized to extract a set of higher-level representations, which are then fed into a convolutional long short-term memory (ConvLSTM) module to construct axial dimensional feature maps. A novel adaptive-weighted cross-entropy loss (ACE) is introduced to optimize the output of the SLP-Net with a view to ensuring that as many valid features from the previous images as possible are encoded into the later CT image. In addition, we employ sequence attention maps for auxiliary classification to enhance the confidence level of the results and produce a case-level prediction. RESULTS: For evaluation, we constructed a dataset of 258 chest CT volumes with 153 VP, 42 BP, and 63 normal control cases, for a total of 43,421 slices. We implemented a comprehensive comparison between our SLP-Net and several state-of-the-art methods across the dataset. Our proposed method obtained significant performance without a large amount of data, outperformed other slice-based and volume-based approaches. The superior evaluation performance achieved in the classification experiments demonstrated the ability of our model in the differential diagnosis of VP, BP and normal cases.

Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods.

The first known case of Coronavirus disease 2019 (COVID-19) was identified in December 2019. It has spread worldwide, leading to an ongoing pandemic, imposed restrictions and costs to many countries. Predicting the number of new cases and deaths during this period can be a useful step in predicting the costs and facilities required in the future. The purpose of this study is to predict new cases and deaths rate one, three and seven-day ahead during the next 100 days. The motivation for predicting every n days (instead of just every day) is the investigation of the possibility of computational cost reduction and still achieving reasonable performance. Such a scenario may be encountered in real-time forecasting of time series. Six different deep learning methods are examined on the data adopted from the WHO website. Three methods are LSTM, Convolutional LSTM, and GRU. The bidirectional extension is then considered for each method to forecast the rate of new cases and new deaths in Australia and Iran countries. This study is novel as it carries out a comprehensive evaluation of the aforementioned three deep learning methods and their bidirectional extensions to perform prediction on COVID-19 new cases and new death rate time series. To the best of our knowledge, this is the first time that Bi-GRU and Bi-Conv-LSTM models are used for prediction on COVID-19 new cases and new deaths time series. The evaluation of the methods is presented in the form of graphs and Friedman statistical test. The results show that the bidirectional models have lower errors than other models. A several error evaluation metrics are presented to compare all models, and finally, the superiority of bidirectional methods is determined. This research could be useful for organisations working against COVID-19 and determining their long-term plans.